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Abstract 



• \ 



semanjiic differential instrument wac developed for student fevalu^-' 



tionjof incjividual /counselipg conference^. 

jury of ^ counselors > cbunselor educators and counselor administra- 



tors confirmed the content 



and construct validity of the* ins t rumen 



Cbnstruj:t valinity was demonstrated^ also^ by, means of aprincLpal 



This 



coii5)onents analysis of 281 students evaluations of 11 counselors. 

} 

' analysis yielded three process and >two outcomes factors which atcomted 
for 60 percent of the total common variance, 

Internal consistenipy reliability coefficients,* which were for 



oti instrument and factor scores > confirmed the multi-dimensionality of 



,the 



instrumei.t, also. 

This instrument £b vked currently , in ojur counselor evaluation system. 

\ 



r Development of a S.ipmantic Differential Instrument for , / 

Student Evaluation of Cpmmunity College .Counseling Conferences 

\ 

The toportance of the counselor's ability to e xamin e, criticize and improve , 
his counseling performance has been expressed by many counselor educators 
(e.g., Boy-end Pine, 1966; Patterson, 1964; Peters and Hanson< 1963; and 
Truax, 1965) • Martin and Gazda (jL970) indicate that the need for a method 
of counselor self-evaluation extends beyond the training program. ^ 

Unfortunately, however, little research on the -measurement of counselor 
effectiveness has been reported (Blocker, 1966; Brown, 1969; and Thorsen, 
iw^. For example, Broro reviewed seven years of the Personnel and Guidance . 
J 6muci i . He napted that although 1^1 articles (or 19Z) were concerned with the 
counseling process, only 36 article^ (or 4.4%) dealt with outcome^;/ Bfown con- 
cluded that relatively little research has dealt with counseling o\itc^es . In 
a recent review of literature. King (1975) stated "As a body, psyclre?.^i^> 
especially professional service pjoirtLders . . . are not conscientious .4iy5Ut aieck- 
ing on the effectiveness of tReir work (p. 3J." "V,v^ 

Furthermore, because it focuses on therapy or counselor training, the majority , 
of literature in this area has littl^^relevance for ev^uating the effectiveness 
of community college counselors^ The instruments developed by T^uax and Carkhuff 
(1966)^d by l^iteley, Sprinthali, Mosher, and Donagh^ (1^67) ff:i example, are 
oripTited toward psychotherapy. -Other instruments and procedure^'^uch as ^ose 
pres^neld by Edfetein (19Z4) and by Silverman and CJuinn (1972) kre designed for 
courselor training. Whereas^^^Chese instruments and approadies*BTovide sugges- 
t?*ons fdr measuring thfe effectiveness pf communis college coUhseiors , none, 
can adopted entirely.* ' 

Although the counselor evalijal;ion instruments developed at a nuni>er of coormii- 
nity colleges were reviewed (Delta College, MI, William Rafney Harper College, 
! iLj Meramec Community College, MO, and Sacramento City Cd^fa^ege, -CA: Kinne- 
brew and Bay , 1973), none of these colleges reported data^.the validity or 
reliability of their instruments. In addition^ the rqle ^ ;th^ counselor, ^ 
defined by these instruments, varied from college to coUe^. Again, while 
suggesting several items for instrument development, the ^ack of data on their 
technical characteristics made tliese instruments inapprjt^Hate fo^r out use. 

Reflecting this larger context, annual student evaluation of counselor effective- 
ness a^ Moraine Valley Community College has been conducted by means of three 
unvalidated instruments.^ As we move toward increased 'accountability througji- 
out -oui college* however, a student services committee has developed a conceptual 
design' for a comprehensive system .for evaluating cotidselor effectiveness. As a- , 
component of the larger system, the puirpose of thi% .project was to develop a 
valid and reliable diagnostic instrument for stUdenj^ evaluation of individual ^ 
counseling co'hferences. " ' . ' - ^ . 



REVIEW OF LITERATURE 

In addition to the evaluation instruments of the conftunity colleges named 
pteviously , the literature reviewed for this project included' books dealing 
vith counseling theory, a DATRIX II search of doctoral dissertations directed 
toward the measurement of counselor effectiveness an"d review of ERIC docu- 
"ntots and journal articles concerned with evaltiation of counselor effective- 
ties$ or the development of evaluatioi^ instniments. 

Ihe literature search fo9a^d on effective co^jnseling processes, anticip^ed 
cou^^ling outcdae^^and measutlhg^ counselor effect iveness". The literature re- 
"A'itiw X5 presented according to thgge~xgp±cgvr:~~ , — - - 



Effective Counseling Processes 



AlthouJ^h extensive, the literature jregaj'ding effective counseling processes 

contradictory. Rowe, Murphy, and De Csipkes (i975) .reviewed research re- 
ported since i960 on the relationship between couns^^lor characteristics and 
coiinreling effectiveness. Although d sizable effortXhas been invested in this 
area, they viewed the results as generally disappoiijti^, of ten ,contr5dl,ctory, 
and only tentative * ^ . r 

On the one hand, Rogers (19&ik)., Snyder and Snyder (1961), Truax (i963), Truax 
and CarkFiuff (1964a) and otSicr^ |lace emphasis/ on the counseling* relationship 
and the need to establish tHi4 relationship wl^th the client before effective 
counseling can take place, iseirez* (1965) asserted that the counselor -can ini- 
tiate, facilitate, ani mainjt^^i^ the interactive process if .he communicates 
feelings of spontaneity aaid' ^^^^mth,, tolerance,^ respect;, and sincerity. In order 
^ to continue the interactive pf^ces^ between the coxmselor and the clifent, 
.Patterson (1966) felt that a Tainimup pf enphatic understanding is necessary: 
, empathy, interest, acceptance," and*|mderstanding are characteristics rtiich 
are essential for influericing otherjs therapeutically. 



should 



In contrast, Rowe, et al. <1975) suggested that the-fpctis of research 
shift from the personality of the counselor to pattictilar behaviors, fekills, 
>v interactions and t]ieir relatioti^i]^ to counseling otitcomes. Emphakis should 
iKangt froii what the counselor is>^<A±ch is often arbijtrarily definedl, to what 
can perform. The authors, reported that initikl efforts at associalting par- 
ticular counselor behaviors Vith.' <;founseling^ outcomes have produced optimistic 
^results.. For instance,' Banks, B^t^ftson, and Carfchuff |[1966) have foupd that 
^clients of high-faqilitativ|, pbupselors engage in significantly more self- 
j exploration than clients, of |.low-|c*acilitative couhselor^- Furthermore, numerous 
investigations (summarized :|>y CaorVhiiff, 1972) successfully demonstrate the sig- 
nificant influence of high ftov^^elor facilitation on several indexes of coun- 
selling outcomes. Carkhpff J^vieiif^^ the high-facilitator as a person who possesses* 
a liirge repertoire of skills catliBr than* one who represents certain abstract 
perionality traits. ^ 

Sev^feral studies have b^in cotts^ijfcted dealing with measured characteristics of 
counselors. A number .djf ^tujjifea reported that the counselor i's person-oriented 
as opposecl to things-briehteQ^ This quality i^ variously referred to as warmth, 
friendliness, altruist:, .irocliLl service, and interpersonal sensitivity by 
Stejfflre, King, and Ujifgre^ (1962), Combs and Soper (1963>, and Wicas and 



Mahan (1966). Receptivity, another trait relevajit to counselors, is reported 
as passivity, conformity, f lexij>ility, or absdhce of dogmatism (Stefflre, F'JLng, 
and Leafgren, 1962; 'Combs and Soper, 19$3; Wicas and, Mahan, 1966; and MillikM 
andTaterson, 1967). 'A third trait counselors share is that of basic e^o 
strength- or self-acceptaiKe, which is identified as emotional stabiJit^^, 
seK-conf idence , self-control, intelligence, or a greater involvement ^with 
reality and problem solving than with one*s own inner needs (StefflrB, King 
and Leafgren^ 1962; Combs and Soper, 1963; and Wicas and Mahaaj- 1966)* 



J)espite the apparent diversity of theories and empirical findings, several 
common themes 
These include 



common themes identifying the characteristics of effective counselors emerged 



— — '- — ^^^ "^ 6 a billrty to comnunicate personal self -acceptance; ' . 

(2) acceptance pf the client; 

(3) receptivity or responsiveness to the client; and 

(4) support of the client . ^ 

Because these *p recesses are applicable to the educational^ vocational and 
personal-social 4o^^ns of the client** s life, they were used Ss^ factors on the 
Moraine Valley student evaluation of counselor instnment to mea^^e counseling 
pp^9cess. Terms suggested from the literature review such as "interested,** 
'^accepting,*' *J<ST5ept:iTC," "sensitive," and "comfort^le" were used as i^ems in 
B^easWing these scales \ 




Anticipated Counseling Outcomes / 

In" recent years, student or, human development has been -employed as a model 02 
philosophy by communit;y college counselors. Brown (1972) defined student develop 
ment in terms of ^^j^t college catalogs and goal statements often describe as the 
"whole student" or the "liSerally educated" person. He noted that most college 
goal statements aspire to promote independence of thought and critical thinking 
in stud^ts; to make students better citizens, as well as to make them more knowl- 
edgeable about their x:ultural heritage/ to help students Wderstand themselves 
and relate better with others; and, of course, to prepare them for a profession.. 

Rogers Cl96lb) indicated that counseling should help the client "become more^ 
self -directing, less rigid, more open to the evidence of his senses, better or- 
ganized and integrated, more similar to the idea which he has chosen for himself. 

Patterson (1966) defined the goal of counseling as "the changing of behavi6r 
or personality in some respect^ or to some exterjt." He also asserted that the 
goal is for the client to becotfe a responsible, independent, self-actualizing 
person capable of determi^ning his own behavior." 

According "to Smith (197A): ' / 

A client is most likely to seek help because of either a problem ^ 
with the self (intetpal conflict) or an environmental concern ' 
(inability to cope effectively with some aspect of external 
re^ity). The task of the counselor— in addition to.communicattSS 
warmth, respect and empathy', to the clitot — is to help the clidnt^ 
discover the spedific probl^ and clearly state his or^ her goij ;^ 
for the counseling process. ^ . * 



The outccjoes of counseling include a wide range of behavior change.^ The 
client should become more independent, or autoncQous; he should develop 
critical thinking abilities, gaining a greater sense of identity asTell 
as relating more effectively with others. The client should became freer 
but also more responsfble; he should become more self-actuaUiing. 

From these rather specific outcomes of counseling, less specific, more 
general outcomes were ^elected for use as factors on the student evaluation 
ot counselors instrument. These are^s include: , ■ 

(1) conference worthwhileaess; ^ - 

' (2) client independence; and. 
(3) jclient self-acceptance^. . ^ 



The l^ter^ture review provided .definitions for these scales. Conference worth- 
whilenesa^is defined as a sityation in which the counselee believes tha,t he or 
she bas heen helped to overcome, obstacles to personal growth (APA, 1956), ' 
Client independence is achieved when tbe client believes he or she has become 
more independent, ^elf-directing, and" autonomous (Patterson, 1^66: Rcigers. 
1961b; and Chickering, 1969). Client self-acceptance is attained as the 
client. becomes more self-actualizing in terms of determining his own behavior' 
(Pattersoir, 1966) , < . . v * • 

Measuring Counselor Effectiveness " ' ^ 

'^ N . \ * ! • 

t "J^^ ^ aPP"ach to the evaluation of\ounseling ^kectiveness emerged ^ 
in the 1960 s. In one form of- this approach, trained observers use carefully 
constructed' scales to rat« counselor performance on the basis of chatacter- 
istics considered essential, for productive counseling. Two such scales are 
the Counselor Rating -St^^le. (CRS) CWhiteley, Spririthall, Mosher and Donaghy, 
1967) ^and the Tr^iac aad ^arkhuf f Scales (1966)^ The CRS ineasures the coun- 
selor s behavior and his responses on proje-cgB -.tests. The Iruax and Carkht f f 

measure different, levels of^ounselorlCnctioning al^mg ^evefaL "core 
facilitating dimensions" such .^a-^athy, respect, genuineness, and excrete-' 
ness. The psychotherapy orientation and trained observer requirement of these > 
rating scales made them inappropriate for this project. 

In another form, of the rating scale approach, clients evaluate counselor I 
effectiveness by recording their perceptions of how often a number of coun- ' 
If^l ^S?;^?^/''''"''* examples of such scales are the Interview Ratine ' 

Anderson, 1962) and the .Counseling Evaluation I n- • 
ffP^^ ] (Linden, Stone and Shertzer, 1965). The IRS consists of 50 
Likert-type items designed to provide an operational definition of the coun- 
seling relationship characterized by "ideal , rapport" between cUent and coun- 
selor. The CEI consists of .21 Likert-type scales, which yield three factor 
scores: counseling- climate, counselor comfort, an'd client sa'tisf action. , 
Although the validity and reliability of the CEI has been established, and' It ' 
V ^r^"^ several studies of the effectiveness of college counselors, 

the CEI factors vere considered neither sufficiently sound theoretically nor 
sufficiently diagnostic for use at Moraine Valley. ' 



Competency-based measurement of counselor effectiveness was introduced by^ 
Vatdval in 1974, '-In his doctoral dissertation Percival Had a jury panel 
c£ counselor educators, consultants and 'public school counselors validate 
a list of 192 rounselor competencies categorized into the 'areas of counseling, 
consultation, coordination and organization and evaluation/ In addition to 
being gMred to the training of public school fiounselors, Percival 's instru- 
to*nt v/j^ not tested in practice,' 



^ ^ • / 




ERiC 



P 



PROCEDURES 



On the basis of this review of literature and of the context of^the problem, 
the Student Evaluation of Counselors (SEC) was constructed, subjects were 
selected, administration and scoring procedures were, developed, and proce- 
dures for establishing instrument! validity and reli^ility were designed. 



Construction 



of t^ 



Student Evaluation/* of t Counsel 



jrs 



(SEC) 



Although numerous formats were cdnsid^red^ for, <iey eloping ^-nstrumenjts, the ser 
mantic differential- was selected Ibecause |Lt jti^ged suf f icj.ent^yjfreliable 
and vali^ for many research purp^fses (Os^if>od, ,Ware, an<^ ^}or^iSy jl9j^lX,. Acjcojrd- 



ing to Kerlinger (1965), it is also fle^ 
vaifying demands, quick and easy to admin 
ential us^s a simple fo^at to measure ti 
various concepts. 



Osgood (I9f)^y suggests th^K threi| prima 
affective meaning. The first, and most 
dimension, the sefond,is called potency , 
Foster (i973) suggest that "altl^ugh man^j 
measure all three factors^ 
before he begins the careful process of 

scales." * This, implies that an iiistnmieni may/bi desigicifed in whic i[ only ttie 
evaluative* factor is measured. Lj\ f act , [jseveraL' evaluative facto fis may b^ 
measured oa^one instrument « 

.The Moraine Valley instxannents U3ed t^e fevaluatJive factor as defiijed by Osgood 
to determine how effectively the counselir utilized thbfee proceissas or persoiality 
characteristics of effective x:ouiseling pndr^furthermotr i, to evjaliate fhe bubcoiaes 
or worthwhileness of the counseling procfess. *^e studeit develbpaent committee 
focused on these two dimensions :>ecknse jtfie former meas irea^ howj a counselor loes^ 
what he or she does, and the latter mea^res the effe^tii of* what the cotinselDr 
does, thus offering a more complete view of 'the counsqlSr in tHe counseling pro- 
cess* Each dimension was further concej^'tualized in terms of saveifal hypothe 
sized ^actors. . ^ 1 \ i . I . ^ 

\ . 

fhe construction of the SEC involved several steps: concept seletftipn, scala 
'selection, quantifier selection ^ format ^development, preparation of instructions, 
counseling services, and summary comments. ] 

"i > ' J 

Concept selection . Chosen on th& basis of Osgood*s (1957) criteria for concept 
selection, the following concepts represented tl^e ^counseling ^tocess aii^d outcome 
dimensiogs: f 
\ . ■ ^ • , f 

(1) '. During the counseling session* my^ counselor was'; 

(2) .\ During the counseling session^ my counselor^ : , ; 

(3) '\ My, sessions with the counselor were - 

(4) \As a result of my counseling session: 



|le and relatirfv^ly easy tjo* adapt ftp 
Iter and scote . The^^^cntic df^fer- 
compl'ex psych( logical c earring ojfi 



facfcots exis£ Lp Jthe (Jottain of 



>ortan^ factor Is called 



Bashopk aid 



id the third I . activity , 
semantic differential instruments 
the investigator should himself define 



pi evaluation 



this constjruct 
ievelopLng appropriate co:vcepts and 



Scale selection . To measure each hypothesized factor, paired words or phrases 
were selected or developed from existing ^instruments and scales according Jto; " 



Osgood's (1957) criteria for scale selection. 



\ 



Quantifier selection ^ On the basis of a number of studies using 'scald 5 with 
various intervals, Osgood (1957) recommefided using seven altemativea,*^"8ince 
with seven alternatives' all of them tend to be used and with roughly, if j not 
exactly, equal frequencies." A seven-point scale permits greater '^iariatltod* in 
respons,es and, therefore, would be a more sensitive measure of change tWan/ 
three- or five-point Scales (KjeYlinger, 1965). ; I 

Typical directions for completing a semantic differential format designate the 
central scale position^'*neutr'al" ,and the extreme positions "very." In/his. in- 
structions, Osgood calls the positions adjacent to the neutr'al central/ pos^ition 
"only slightly related" and the positions between these and the extieaes "quite 
closely related" (1957). * /- ' 



SEC quantifiers listed below thus parallel Osgood's seven alternative p^itions: 



VERY 



QUITE- 



SLIGHTLY 



DOESN'T 
APPLY 



SLIGHTLY 



QUITE 



VERY 



Since not all clients would pierceive all items as being applicable to their par- 
ticular counseling situation, "doesn't apply" was .substituted for "neutral." ' 

FotTnat, deyelopmeiii . Adapting Osgood 'i (1957) recommendation to place each con- 
cept at tfie top df a separate page, the dimensions of counselling process and 
counseling outcomes were therefore assigned separate pages. The order of factor 
representation within a dimension and* the polarity of scales were randomly assigned 
Coiti-tosslng (Hecnt, 1970) vas used to 'vary the' order of both ^factor representation 
and polarity of items whenever possibly*.. As a result of the wording of scales 
representing the factors of counselor support' and worthwhileness, they were not . 
randomly assigned Jn the instrument. The polarity of items to measure .these conp 
cepts were randomljj assignedv^ujwever. . . 

For each of the thirty ittos, one^ adjective was placed at each eid of an imaginary 
h6ri2ontal lihe across the page. The seven quantifiers were underscored at the 
top of each set of adjectives, and a seri^Bs of zeros and dots werfevAXSed to vepr^" 
serit the quantifiers on each horizontal' iXne. \\ * ' ' * 

Preparation of instructions . Separate directions were developed for the Counsel- 
ing process and counseling putcomes portions of the ins,trument. For the former, . 
respondents were insbruoted to place an "X" at the point which represente.d their 
judgment of the cqunseling methods used by the counselor; an example was given, 
'The directions for the counseling outcomes portion of the instrument asked the 
respondents to pj.ace an "X" at the point which represented their, feelings about 
the results of their counseling sessions; no example was given. 

Counseling secVtces . Since counseling viewed as encompassing the educational, 
'vocational, and personal-social dotnairls of the client's life, •he specj^fic ser- 
vices provided by counselors veye^evaluated also. Items peasure Serviced, 
such as giving accurate transfer information, w^re included on the final page 
o^ the instrument. Using the tikert scale below, respondents, were asked to re- 
spond to the stem "This counselor provided service" by J " 



STRONGLY 
AGREE 



MODER- 
ATELY 
AGREE 



SLIGHTLY 
■AGREE 



JSN'T 
?PLY 



SLIGHTLY 
-DISAGREE- 



M(^DER- I 
ATELt 
DISAGREE 1 



Strongly 
disagree 



S^nnmarv Comments . Consistent with thfe views of Warwick and Lininger (1975) t^jo 
open-ended questions were constiructed for the final portion or the SEC. They 
were designed so that "the respondent <;an follow, his own logic and chains of 
association, free from the constraints of an imposed spheme/' In tune with the 
purpose of the' instrument , the questions' asked Respondents to answer regarding 
areas of strength and potential improvement for the counselor. A sample^SEC 
instnmient is presented in Figure 1. • * • * 



Insert Figure 1 about here 



Subjects, Administration and Scoring of SgC 

Subject selection and administration of SEC . Records of contacts with all 
students seen for individual counseling 3ppointments from January 1974 to Decem- 
ber 1974 were maintained by ten counselors through ^the use of JBM cards. Thp 
student evaluation of counselors instrument was mailed during the spring of 1975 
to these students, approximately 600 in nuinber. Instrumen|:s were returned by 
210 students who had been seen once or twice and 71 students who tiad been seen 
three or more times by ^counselors at Moraine Valley • This initial mailing was • 
designed to prgvide intormation on the validity ^and reliability o!E the 'instrui^t* 

^ Scoring . - F.o^lowing th'e common practice initiate'd by Osgood and llis associAt^s 
(1957), response posii 
from one for tjhe lessi 



ions for each item were assigned integer values ranging 



favorably pole td seVeri for the Aore favorable pole, 
j.f I ' ' J ' 
coded fot keypunching by a Molbaine Valley 



choicesSy^^e 



The student response 
clerk-typist wio usee 

of student responses for eaW item -(Hechtv, 1970). Eac& numferical value, was sub- 



tranSpariiht scoring masks to deHerminW :he numerical value 



sequently^ecojrjded in an appi^o 



riate box on a data coding sfiejpt 



For each sJttSdent, 4Ioraine Vailey keypurtch operators punched mid verified ^ IBM 
card containln^cpunselor desigijation anB other 'variables (Hi^cht, 1970). 



Because factor scires are lessi numerous ^nd have been shown .tip' be more reliable 
than individual idem scores (Norm^, 1959*), they \i[eie computed to tepres.ent each 
student's "scores'! on , the SEC* K^e (196|9) ^studied the validjlty op the as,sumed 
factor structure ot fourteen, adject J^ve sc^ales usea to judge a numbler of concepts. 
He concluded that |erroneous res(ults and Conclusions would be genejrated by scor- 
ing and analyzing responses based on^asfeuttted scale performance^. ^In addition, 
researchers. and practitioners haye been ak^ksed to specify logically (Heise, 1969) 
and investigate emp|.riGally (McKie*and Fo|ter, 1972) the ^Iructur^ of the condept 
domain under consideration. Therefore, SkC factor scores were computed dn the 
basis pf factor analyse of S'EC data raVher than on "the basis of the hypothe- 
si2e*d concepts* \ . \\ ' , ^ 



Total. score, dimensibn scores, and factor &cohM were qalculdted for each stuSent; 
'by , summing the weighted responses for appropriii^e items (Osgood, et al., 1957) 
The r^spoiv^e oi^ each item. was termed the scale score. A summary sheet- containing 
the average, total,, dimension, and factor scores |£epared for each cdunselor. 
It also contained information regarding th^ number of ^respondents and the pern 
centage of agrejement with each of the quantifiers related to counseling service^. 



For each counselor and for a composite^ of all' counselors, .average scores of 
each kind were calculated. An SEC scale score profile sheet w.as cons^tructed 
to petTiiit comparison of the individual, counselor's average item score profile 
with the composite average item acbte profile for all counselora. 

Validation and Reliability Procedqjj^es for SEC * , ^ 

• . * ^ 

Statistical, ^nd. logical procediif!6's we^ e. used to validate t?he SEC, Following the 
recommendation of Bashook and Foster (1973), resjJanses from all 28}. students 
were pop led for a ptincipal" components analysis to evaluate the copstmct validity 
of SEC. Content and construc^validity were also evaluated iby a jury panel con- 
slstfiig" of four' cbuhselor ^ducatprs , four ^ couuu& nity coiiege cotAiseior a3miTiis= 
trators.and four comminrr t-y^ rM l^gp mtmg e 1 o rg^ The counselors to be evaluated . . 
by the SEC^comprised an additional "ad hoc" jury panel, A sample of the.ques^. 
tionnaire completed by the jury panel' is presented in Figure 2, 



\ ' Insert Figure 2 about here 

Internal consistency reliability coefficients were calculated for factor scores 
and the total score by means of Cronbach's Alpha. Point biserial correlations, 
were calculated to evaluate the relationship between item and factor scores and 
xtdm and total scores. . ^ 

All statistical calculations were performed on an IBM 370 Computer using program^ 
FACTOR and TESTAT (Veldman, 1967, pp' 174-180 and 222-2^36). 
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- • . i\ ' . ^ 

^-^ .^ ' \ RESULTS AND DISCUSSION 

Validity of- the S~EC .J * . 

Factor analysis . - ^Principal components analysis of pooled resj)onses, with vari'^ ^ 
max rotation to simple structure, yielded three process and tyo outcomes factors 
which accounted'for 60 percent of the total comnon variance. As shown in Table 1, 
the factor loadings sug'gested these factor descriptions: Cis^j^erence Worthwhile- 
*ness, Client Independence, Acceptance of Client, Counselor Se\f -Acceptance and 
Counselor Support. ^ ^ > 

These empirical resets agreed closely wi^ the intended results of the 'SEC. 
As shown in Table 2, seven factors were initially hyi)othe6ized by the student 
service 'committee, and five were identified through the factor analysis. Sig-V 
^ nificantiy, two factors, "Counselor Support" (factor 5) and "Conference Wortl^^;^ ^ 
whileness" (factor 1), were identical to those hypothesized bji^the student 
development committee. Among the other five hypothesized factOTSvo^e, "Re-- ' 
sp.onsivenees to Client" (factor 6} , splitfto become part of two process factors 
"Accept^mce of Client" <faGtor(3) and "Counselor Self-Acceptance" (factor 4); 
and two, "Client -Self-Accept ancej* (factor 8), and "Decision-Making Ability" 
(factor 7), became one outcomes factor "Independence" (factor 2). , * 

In addition, because no scales crossed over between the process and ox^com&s 
dimensions iti this analysis," it was in^rred that students perceived Iprocess 
and outcomes as separate and distinct dimensions. This analysis al^p identified 
three scales which students apparently perceived in different ways. 

A second principal components analysis was calculated separately for thi process 
•and outcome scales* to establish the validity of the factors within each dimension. 
Although ^is analysis yielded the same five factors^ shown in Table 1, it con- 
.ficmed the ambiguity of three scales: "dpen/Closed^»»'"I feel O.K. /I feel not O.K.,'" 
and "Encouraged me to express ideas /Discouraged me from* expressing ideas." These 
' scales were deleted from subsequent analyses ♦ 

♦ • ' * ' ' 

Jury 'panel analysis . Results of jury panel responses are presfented in the order 
' of items on the 'jury panel questionnaire (Figure 2). In response .tb the first 
item, jurors indicated that directions were (pearly worded. The percent of ju- 
rors designating the clarity of eacK SEC ited is shown in Table 2. . Although 
all items met the minimum standard of 70% agreement, some judges rated some 
r items unclear to them. * * ^ ' , 

I -*^*"^Ta}>le 2 alfso shows jury panel agreement on the dimension and factor location of 
I SEC items. Judges agreed on the dimension location of 29 of 30 SEC items^ Fur- 

thermore, their dimension classification agreed' exactly with those ^ the student 
.^rvices committee and with the results of the principal components analysis. 
M least 70 percent of the jurors agreed on the factor placement of 15 of the • 
final 25 SEC. items; a surprising result considering that no definitions of . ^ 
^ "factor names (Were given to the jurors. Table 2 also shows that juror platfe- ^ 
ment of itemi on factors was more like the placement of the^student servicea* 
committee than like the placement by factor analysis. . • ^ 

. . However, the consistency between counseling theory, the structure hypothesized 

by the student services committee, the confirmation by the principal cpii5>oneffts 
^ analysis ajid^the jury panel comprise evidence of the construct validity of the 
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Evidence of the content validity of the SEC^was provided by the jury panel, also. 
Each SEC factor was rated either consistent 6t very consistent with counseling 
theory by at least 70% of thfe jury panel. In addition, 75% of thQ jury panel 
rated the SEC dimensions apprx>priate or very appropriate for evaluating coun- 
selor serviced. In Uieir open-^nded comments, jurors complimented the compre- 
hensiveness of ^We SEC and rated itf^an appropriate way to assess- counselor 
^effectiveness. . 

^ Reliability of the SEC " « . , 

Item reliability . Item reliability in relation to total and factor scores, 
were calculated using point bl!serial correlaMdns. As presented in Table 3, 
for every scale, the correlation between item and -factor score w^ higher than 
the correlation between item and total score. Thus, the instruilient is multi- 
dimensional. ^ 



Insert Table 2 about here 



Item-factor reliabilxties r^ged from a high of .87 to, a low of .66. Using .70 
as a minimum acceptable internal consistency coefficient (Tuckman, 19.72), 
i-tems 6,^ "honest /disho;nest" and 8, "attfentive/inattentive" were eliminated" 
from further analyses. " * 



Reliability of totals dimension and 'factor scores . Deletion of three ambiguous 
and two unreliable items reduced the number ^,o^ SEC items from 30 to 25. Factor, 
dimension, and total scores, t|ieref0re, we're calculated over 25 items. 

determine whether SEC scores were sufficiently free of chance errors to per- 
init use of total, dimension^and factor scpres,, internal consistency reliability 
coefficients, were calculated using Cronbach^s Alpha. As shown in Table 3, the 
total score had the highest reliability and the outcome dimension was slightly 
more reliable than the process dimension. All of the composite score reliabili- 
ties exceeded the .70 minimum a<!cep*table level. The standard exror of a total 
score was 5<points out of a maximum possible score *of ,245. These analyses confirm 
the reliability of the SEC for practical use. 

T^le 3 , • " 

Alpha Reliability Coefficients for SEC Composite Scores 

TOTAL SCORE 9^ 

DIMENSIONS— 

y Process 1 ^ 87 ' * 

^ Outcomes. r * . . . . i.^. 90 % 

FACTORS— . . , ' ^' 

Acceptance of Client....* * ^ 85 

JSuppbrt of Client ^ 69 

Counselor Self -Acceptance. 74 

Worthwhileness* ^ .90 i 

.Client Independence • ♦ .85 • . 

1 A • * ' 



, . Sin*IARY 

» 

Initial developi^nt of a semaatic differential iK^rument fot student evaluation 
of counseling conferences was described in this p^ex,, I 

Eased upon jury panel and principal con^onents* analysis, the SEC shows /substan- 
tial content and construct validity. In addition, SEC total, dimension anji 
factor scores ^have internal consistency reliability coefficients whicll are com- 
parable to those reported for many commercially published standardized tests. 
The validity and reliability of SEC factor scores permits their diag^^ostic use 
in 'assessing the effectiveness of counselor performance on the proc^^ and out- 
^s dimensions. 



Limitatrons in the initial development of the SEC include; 

. .Use of only 10 counselors ih one institution as the sample 
evaluated 

• Use of only 281 student evaluators- 

. Lack of norms or performance standards for intetpretatiyon 
of results , 

i ... 

All of these limitations wilj be eliminated as further development occurs. 
KeanwffiLie, ^en in its initial. stage, the SEC represents a positive contribution 
to the develbpifient of a conprehensive system for the eva^ation of counselor 
effectiveness at Moraine Valley. 
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MORAIHE VALLEY COHMOHLTY COLLEGE 

Figure 1 \ Ik 

student Evetluation of Counselorg 



Counselor's Nana: 



foAt I. C6un6tUng ?Ajoc£s6 ■ » _ . i 

Directions: Tx>r each pair of words or phrases, place an "X** at the poinjj 
which represents your judg;eiaent of the counseling loethods 
by this counselor. For example , if you feel that yoxir couns 
was very supportive, KMxk yoxir paper like this: 



, Doesn't 

Very Quite Slightly '"Apply 



supportive ^ 



Slightly Quite Very \ \ 

o O unsl^)portive 



VuJung tht coun6ztLng 6t&6ion, my coan6zlo^ um: 

'\ Doesn't 
1 Very Quite Slightly Apply Slightly 



QuitH^ Very 



1 confusing 


0 


o 


indifferent 


0 


o 


comfortable 


0 


' o 


non- receptive 


0 


o 


accepting 


0 


o 


dishonest 


0 


o 


tuned out 


0 


o 


attentive 


0 


o 


tense 


0 


p 


• personal 


0 


o 


closed 


0 


o 


sensitive 


0 


o 



o 


0 


connunicating 


o 




interested 


o 


xmcoznfortable 


o 


- 0 %\ 


receptive 


o 


0 


rejectii)g 


o 


* 0. 


honest* 


o 


0 . 


' tuned in . 


o 


0 


inattentive 


o 


0 


relaxed 


o 


. 0 


ic^rsonal 


o 


0 
• > 


open » 


o 


7^ 0 

e 


insensitive 



VanXng tht courts tUng 6t&6ion, my cotui6zto^ 



Strongly Moderately Slightly Doesn't Sli^tly 
Agree Agree - Agree ^ Apply Agree 



Moderately Strongly 
, Agree Agree 



discouraged *ine 
from expressing 
Ide^ 

offered - 
alternatives 

encouraged 
iitdependence 

avoided xsy 
feelings 



encouraged me 
to deal with 
problem 



o^ 



encoTiraged ine 
to expres<^ 
ideas , 

dictated the 
solutions 

encoxiraged 
dependence 

helped me to 
t9g>ress ^oy 
feelings 

avoided my 
problem / 



?cuU n. CounAtting Owtcomu 

Directions: Fpr each pair of words or plirases^ place an '*X*' at the point vrfiicK 
represents youg: feelings about the results of your counseling 
sessions with this counselor.* 

Mi/ 6Mions mXh thz coun6ttoA, weA^e: ' . 

Doesn't 

Very Quite Slightly ' Apply Slightly guite Very 



o 
o 



neaningless O 

wo^hless O 

helpful O 

unsatisfactory o' 



a Kt&uJU oi my cbun&zting 6t&6ion: 



o 
o 
o 



o 
o 




strongly Moderately Slightly lX>esn't Slightly Moderately Strongly 
Agree Agree _ Agree j Apply Agree Agree Agt^ 



decj.sions wete 
made for me 



I have a more 
positive .atti- 
tude about 
myself 

I was helped 
to see my 
problem 

I feel shut 
off. . 



I learned new 
ways of 
behaving 

I was encouraged 
to think for 
myself. 

I feel not O.K. 



O 
O 



I *would refer 

other students 

to this counselor O 



X have less con- 
fidence in my 
abilities and 
decisions 



P 

o 



/ 



o • 
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X learning 
O to make my 
own decisions 

X have a more 
negative at- 

Jtitude' about 
O -^^piyself 

' I was not 
helped to see 
O my problem 

I feel I can 
O r«tum to my 
counselor 

I ''did not 
Team new ways 
O of behaving 



I received a 
great deal of 
eidvice 



O 
0 



r fe^I O.K. 

X would not 
refer other 
students to 
this Counselor 

X have more 
confidence in 
VQT abilities 
and decisions 



?<uit*Ui. CounAtUng ScJtvicu ^ 



DiiUctions: Please place an '^Z*' at the point which represents your judgexnent of * 
the counseling services provided by this counselor*! 

i' strongly Moderately SligliMy Doesn^t Slightly Moderately Strongly 
1 Agree Agree I Agree ^ply ^ Disagree Disagree Disagree 



giving accxir^tc information 
on MVCC courses r. 

giving accurate information; 
On MVCC certificate and 
degree requiretaents. 

giving^ accurate transfer 
information 



0 



o « 



0 •> 



giving adequate assistance 
with my career planning 

adequately assisting me 
with my personal concerns 

directing roe to appropriate 
au^o- visual and printed 
information 



referring me to an appro- 
priate staff person who 
could help roe ^ 



of 



\ 



Directions; Please answer in th^ spaces below. You may want to describe jsix incident 
to. illustrate your cosraents. Use other side of paper, if inecesslary*. 

1, What contributes most to thi« counselor's ef ^eetiveness2 . \ A : - 



\ 



2, In what ways could this counselor increase his/her effectiveness? 



THAMK you FOR VOUR CONTRIBOTION 
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Figure 2 

QUESTIONNAIRE 
. COUNSELOR EVALUATION INSTRUMENT 



PART I. Student Evaluati"on of Counselors Instrument 



The Studejit Evaluation of Counselors instrument is to be completed by 
Students who have seen a counselor for individual counseling. Please 
indiqate your response to the following items regcirding this instru- 
ment, ' « 



Underline any directions on the instrument which are not clearly 
worded. / 

Please c^ircle the numbers of the items on tl^e Student Evaluation of 
Counselors instrument which are not clear in th^lr meanjng. 

1 2 3 4 5 6 7 8 9 id 11 12 13 14 15 16 17 18 19 20 21 

22 '23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 



The Items for the Student Evaluation of Counselors instrument were 
divided into two dimensions: Process and Outcomes. These dimensions 
were divided into five scales: Counselor Acceptance of Counselee, Counselor 
Support of Counselee, Counselor O.K.ness, V/orthwhileness and Counselee In- 
dependence. The names of the dimensions and scales are listed below. Please 
write the numbers of the items (1-30) which in your judgement fit each 
dimension and scale. ^ y 



DiMENSION—Counseling Proqess 



DIMENSION—'Counseling Outcomes- 



SCALE— Acceptance of Counselee 



SCALE— Counselor Support 



SCALE— Counselor 'O.K;ness 



Ik \ 

SCALE-^-^Worthwhi leness 



23 



•SCALE— Counsel ee Independence 





\ 




1 . ■ 

What additional scales, should be considered for use on the instrument?. 


:"\ 


What additional items should be considered for use on the 


ijistrument? 




• 






Please indicate the degree to which the scales are or are 
with your perception of counseling theory • The key is .as 

< * * 


not consiste 
follows: 


\ 



Very 

Consistent" Consistent 



Neutral 



0 \ 
t ■ 



0 



Inconsistent 

0 



of/( 



Acceptance of! Counselee 
0. ; ' 0 



0. 



Counselor Support 



Counselor O.K.ness 




Very 

nconsistent 



0 I 



9 



Worthwhileness 



Counselee Independence 

1 N 
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The model ^ for this instrument is 
jand counseling services ^e'evalua 
priateness\ih evaluating counsel^* 



Very 

Appropriate \ Appropriate / 



tb«t counseling process, coiun^seling outcomes 
ed. Please indicate the degree of appro-' 
ng services on this instrument 



^eut^al\ 



Very 

Inappro^iate Inappropriate 

/ 



ieh woiil 



Please make any qomments which would support your judgement. 





1 r 






' — ^ 

\ i 






1 . • 



' Part II. Additional Comments 

Please make any additional comments regarding the construction, use or 
appropriateness of this instrunBatj. 








-THANK YOU FOi(:VOUR ASSISWNK. .* , 
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Table 2 ' • 

SEC Item Validation by Jury Panel, Student Sexrvicfes Coinmijtee 
and Principal: Components Analysis 
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Table 3 

P.oisk-Biserial Correlations / 
•Between. SEC Itefas and Total and Factor Scores 
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